Recent years have seen increased attention in the area of image compositing—the process of overlaying a masked foreground image on top of a background image to form a new composite image. In general, compositing images can be difficult due the foreground object from one image coming from a different scene than the background image, which leads to conventional photo-editing systems creating inconsistent and unnatural composite images. Indeed, the often foreground object mismatch the background scene in ways that negatively affect the realism of the composite image. For instance, the foreground object can mismatch the background in both appearance (e.g., lighting, white balance, and shading) and geometric perspective (e.g., changes in camera viewpoint and object positioning).
Some photo-editing systems have focused on the task of improving the appearance of composite objects to enable users to create more-realistic looking image composites. In particular, these photo-editing systems have employed Poisson blending and deep learning approaches to automate appearance corrections (e.g., contrast and color saturation). However, these systems do not address the issue of geometric realism. Thus, many photo-editing systems still require a considerable amount of human intervention using photo-editing software. Further, because of the difficulty of understanding and applying proper geometric perspectives, even after human editing, traditional composite images often appear unnatural.
In addition to photo-editing systems, recent direct image generation systems have attempted to create composite images using direct image generation, which includes a pixel-level prediction. Not only do these recent direct image generation systems require large amounts of memory storage and increased processing resources, but these systems are also limited to specific restricted domains, such as creating composite images of faces. Further, these systems often operate at low resolutions due to large processing requirements and finite network capacities. Thus, even these recent direct image generation systems fall short of providing realistic image composites efficiently and efficiently.
These and other problems exist with regard to automatically generating realistic composite images.